Modeling higher-level metrics from graph data derived from already-collected but not yet connected data

ABSTRACT

Systems and methods for modeling higher-level metrics from graph data derived from already-collected but not yet connected data are disclosed. A method includes extracting a first set of actor-related data, a second set of object-related data, and a third set of temporal data from the set of the already-collected but not yet connected data representative of a unit-level contribution to the target activity. The method further includes generating graph data for a graph using the set of the already-collected but not yet connected data, where each of the plurality of nodes of the graph corresponds to the actor or the object, and where an attribute associated with each of the plurality of edges of the graph corresponds to a measurement associated with the target activity. The method further includes modeling a relationship between graph attributes associated with the graph data and a higher-level metric associated with the target activity.

BACKGROUND

Aggregated data related to the users' activities can provide greaterinsights than the data related to an individual user's activities. Thecollection of such data, however, may still incur runtime costs aboveand beyond those necessary for the normal operation of the system.

Moreover, the analysis of the collected data may include techniques thatfocus on metrics that do not accurately capture the complexity of theuser's activities and their contribution to the value of such metrics.As an example, analyzing collected data through attribute-basedaggregations (e.g., sums, counts, or averages) elides theinterdependence of users' actions and the importance of structuralrelationships.

SUMMARY

In one example, the present disclosure relates to a method implementedby a system comprising at least one processor and at least one memory.The method may include collecting data for a target activity relating toan actor or an object during normal operation of a cloud computingplatform to generate a set of already-collected but not yet connecteddata and storing the set of the already-collected but not yet connecteddata in the at least one memory. The method may further include usingthe at least one processor, extracting at least one of a first set ofactor-related data, a second set of object-related data, and a third setof temporal data from the set of the already-collected but not yetconnected data representative of a unit-level contribution to the targetactivity. The method may further include using the at least oneprocessor, generating graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor or the object, and where anattribute associated with each of the plurality of edges corresponds toa measurement associated with the target activity during a temporaldimension of interest. The method may further include using the at leastone processor, converting the graph data into metric space data using agraph embedding process and storing the metric space data in the atleast one memory.

In another example, the present disclosure relates to a methodimplemented by a system comprising at least one processor and at leastone memory. The method may include collecting data for a target activityrelating to an actor or an object to generate a set of already-collectedbut not yet connected data and storing the set of the already-collectedbut not yet connected data in the at least one memory. The method mayfurther include using the at least one processor, extracting a set ofactor-related data from the set of the already-collected but not yetconnected data representative of a unit-level contribution to the targetactivity. The method may further include using the at least oneprocessor, generating graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor, and where an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest. The method may further include using the at leastone processor, converting the graph data into metric space data using agraph embedding process and storing the metric space data in the atleast one memory.

In yet another example, the present disclosure relates to a systemcomprising at least one processor and at least one memory. The at leastone memory may include instructions configured to, when executed by theat least one processor, collect data for a target activity relating toan actor or an object to generate a set of already-collected but not yetconnected data and storing the set of the already-collected but not yetconnected data in the at least one memory. The at least one memory mayfurther include instructions configured to, when executed by the atleast one processor, extract a set of actor-related data from the set ofthe already-collected but not yet connected data representative of aunit-level contribution to the target activity. The at least one memorymay further include instructions configured to, when executed by the atleast one processor, generate graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor, and where an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest. The at least one memory may further includeinstructions configured to, when executed by the at least one processor,convert the graph data into metric space data using a graph embeddingprocess and storing the metric space data in the at least one memory.

In another example, the present disclosure relates to a methodimplemented by a system comprising at least one processor and at leastone memory. The method may include collecting data for a target activityrelating to an actor or an object during normal operation of a cloudcomputing platform to generate a set of already-collected but not yetconnected data and storing the set of the already-collected but not yetconnected data in the at least one memory. The method may furtherinclude using the at least one processor, extracting at least one of afirst set of actor-related data, a second set of object-related data,and a third set of temporal data from the set of the already-collectedbut not yet connected data representative of a unit-level contributionto the target activity. The method may further include using the atleast one processor, generating graph data for at least one graph havinga plurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor or the object, and where anattribute associated with each of the plurality of edges corresponds toa measurement associated with the target activity during a temporaldimension of interest. The method may further include using the at leastone processor, modeling a relationship between graph attributesassociated with the graph data and at least one higher-level metricassociated with the target activity.

In another example, the present disclosure relates to a methodimplemented by a system comprising at least one processor and at leastone memory. The method may include collecting data for a target activityrelating to an actor or an object to generate a set of already-collectedbut not yet connected data and storing the set of the already-collectedbut not yet connected data in the at least one memory. The method mayfurther include using the at least one processor, extracting a set ofactor-related data from the set of the already-collected but not yetconnected data representative of a unit-level contribution to the targetactivity. The method may further include using the at least oneprocessor, generating graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor, and where an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest. The method may further include using the at leastone processor, modeling a relationship between graph attributesassociated with the graph data and at least one higher-level metricassociated with the target activity.

In yet another example, the present disclosure relates to a systemcomprising at least one processor and at least one memory. The at leastone memory may include instructions configured to, when executed by theat least one processor, collect data for a target activity relating toan actor or an object to generate a set of already-collected but not yetconnected data and storing the set of the already-collected but not yetconnected data in the at least one memory. The at least one memory mayfurther include instructions configured to, when executed by the atleast one processor, extract a set of actor-related data from the set ofthe already-collected but not yet connected data representative of aunit-level contribution to the target activity. The at least one memorymay further include instructions configured to, when executed by the atleast one processor, generate graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor, and where an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest. The at least one memory may further includeinstructions configured to, when executed by the at least one processor,model a relationship between graph attributes associated with the graphdata and at least one higher-level metric associated with the targetactivity.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example and is notlimited by the accompanying figures, in which like references indicatesimilar elements. Elements in the figures are illustrated for simplicityand clarity and have not necessarily been drawn to scale.

FIG. 1 shows a diagram of a system environment for collecting data inaccordance with one example;

FIG. 2 shows a diagram of a cloud computing environment for collectingdata in accordance with one example;

FIG. 3 is a block diagram of a system for performing methods associatedwith the present disclosure in accordance with one example;

FIG. 4 shows a block diagram of a memory including modules withinstructions for performing methods associated with the presentdisclosure in accordance with one example;

FIG. 5 shows a node-link graph in accordance with one example;

FIG. 6 shows four different views of a graph that reveals thecommunities of users with strong collaboration in accordance with oneexample;

FIG. 7 shows a block diagram of a memory including modules withinstructions and data for graph embedding related methods in accordancewith one example;

FIG. 8 shows a flowchart corresponding to a method including graphembedding in accordance with one example;

FIG. 9 shows a process flow in accordance with an example;

FIG. 10 shows a search engine example having a user interface, includinga search window, in accordance with one example;

FIG. 11 shows a transition graph in accordance with one example;

FIGS. 12, 13, and 14 show diagrams related to search recommendationsbased on graph embedding in accordance with one example;

FIG. 15 shows a graph that represents the email exchanges among varioususers of an organization as an organization structure;

FIG. 16 shows a table illustrating adoption of a feature related to aproduct over time in accordance with one example;

FIG. 17 shows community structure views that show the adoption of thefeature over time in accordance with one example;

FIG. 18 shows a graph of various communities at a workplace inaccordance with one example;

FIG. 19 shows diagrams of a conventional organization design and theorganization's functional reality based on a graph induced from emailtelemetry data, which may be a subset of the already-collected but notyet connected data, in accordance with one example; and

FIGS. 20 and 21 show a framework that illustrates certain attributes ofa workplace in accordance with one example.

DETAILED DESCRIPTION

Examples described in this disclosure relate to graph embeddingalready-collected but not yet connected data. Additional examples relateto applying machine learning to the graph embedding related methods.Certain examples in the present disclosure relate to interpreting,diagnosing, predicting, and acting on metrics by modeling activity ofinterest as a graph. Such graphs enable automated inference andinteractive exploration of the key relationships between the structureof activity and the high-level metrics that measure certain outcomes.Compared with the common practice of monitoring metric averages overtime, the methods and the systems described in the present disclosurereveal actionable insights into how to transform the structure,dynamics, and outcomes of activity for the better.

Traditionally, entities have used metrics to abstract the low-leveldetails of business-related activity into high-level indicators ofperformance. Such activity of interest may be internal to the entity(e.g., rates of production), external to the entity (e.g., rates ofsocial media mentions), or at the boundary of the entity and itscustomers, clients, or users (e.g., rates of service use). The reductionof such activity to high-level metrics such as numerical averagesignores the connected structure and temporal dynamics of activity ascaptured by high-fidelity logging (e.g., through cloud-basedinstrumentation and telemetry). This is a problem because the use ofaveraging can be both inappropriate and insufficient for a user's needs.

While such averaging may be appropriate when the activity of interest isindependent and identically distributed, this is rarely the case. Formany activities, the actors in an activity system comprise a network andare subject to network effects (e.g., positive network externalities inwhich each additional actor confers benefits to the existing actorbase). Actors are more likely to influence their direct connections thanmore distant nodes, leading to concentrated rather than dispersedeffects. The resulting interdependence of actions violates theassumption of independence. Similarly, the Pareto principle thatcharacterizes the distribution of activity outcomes—that 80% of theeffects come from 20% of the causes (e.g., the actions ofactors)—conflicts with the assumption that activity is identicallydistributed across the population. Averages can also mask significantvariation in activity over time, e.g., a stable number of active userscan result from both a stable user population and a high-growth,high-churn dynamic resulting in a similar-sized population ofever-changing users over time.

In addition to masking the underlying variation in activity, metricaverages only measure activity outcomes and not the interconnectednessof the actions that contribute to those outcomes (e.g., as in the viralspread of ideas or the diffusion of product adoption). Without ananalytic model of connected activity in terms of both its topologicaland temporal structure, it may not be possible to diagnose the specificcause of a given metric value or to infer general relationships betweenhigh-level metrics and low-level qualities of the underlying activity.Such understanding is critical for determining how to intervene in waysthat might transform the structure of activity, and thereby the metric,for the better. Example interventions that operationalize the results ofanalysis include offering subsidies and other incentives to improvesales metrics, running campaigns targeting specific influencers toimprove marketing metrics, and changing organizational structure toimprove collaboration metrics.

This problem is a general one, applying to all metrics where activity isreduced to averages across four classes of socio-economicactivity—interaction, production, consumption, and exchange. Examplesinclude interaction metrics like the number of messages sent andreceived; production metrics like the number of tasks completed orproducts manufactured; consumption metrics like the number of articlesread or videos watched; and exchange metrics like the number or value oftransactions completed. In other words, it affects any business thatuses metrics to set goals, track performance, and make strategicdecisions.

A key use case in which metric averages may be used to describe complexpatterns of connected activity is for businesses offering cloud-basedsoftware as a service (SaaS) via subscription or freemium businessmodels. Such businesses use engagement metrics to track the scale,stickiness, and attrition of service usage. Scale metrics are those thatmeasure the number of active users over various time periods, such asDaily Active Users (DAU), Weekly Active Users (WAU), and Monthly ActiveUsers (MAU). Stickiness metrics examine ratios of scale metrics to showhow often users return to the service; for example, a DAU/MAU ratio of0.25 can be interpreted as users engaging with the service on one dayout of four on average. Attrition metrics are measured as user retentionpercentages over various time periods (e.g., Day 1, Day 2, Week 1).These metrics are all averages, yet for many such businesses, usersbecome and remain active in ways that depend on one another individually(e.g., interacting with specific users) and collectively (e.g.,leveraging community-generated content). The resulting activity networksare often fundamental to business success, but the successful growth andmaintenance of these networks is often dependent on a small proportionof “influencer” users who are frequently active and highly connected(following the Pareto principle). A new class of structuredmetrics—e.g., relating the number, ratio, and lifetime of active usersto their community participation or proximity to influencers—would helpbusinesses to understand not just how their metrics vary and why, buthow their metrics could be transformed for the better by intervening inthe activity.

Certain examples of the present disclosure relate to modelingbusiness-related activity as the connected actions of interdependentactors and to characterizing the structure and dynamics of activity ingraph-theoretic terms. Graphs (otherwise known as networks ortopologies) are fundamental mathematical structures that model pairwiserelationships between objects. They are comprised of nodes (otherwiseknown as vertices or points) connected by edges (otherwise known aslinks or arcs). Families of graphs include heterogeneous graphs withmultiple node types; multivariate graphs with multiple edge types;attributed graphs with attributes associated with nodes; weighted graphswith numerical edge attributes representing the degree of relatednessbetween linked nodes; dynamic graphs with temporal edge attributesrepresenting the onset and duration of relationships between linkednodes; directed graphs with explicit edge directions from source totarget nodes; and undirected graphs with edges equivalent in bothdirections. Any given graph can contain elements from multiple suchfamilies.

In the examples associated with the present disclosure, the actions ofactors are represented as edges in a graph that connect nodes(representing the actors performing those actions and the objects beingacted upon). Such edges are necessarily directed because the actor isperforming the action on the object and not vice-versa. The resultinggraph is also heterogeneous in structure, representing relations betweentwo distinct types of nodes (actors and objects). For many forms ofanalysis, however, it is beneficial to operate on an undirected graphstructure showing relations between nodes of only a single target type(e.g., actors or objects, but not both). This can be achieved bycreating a new graph containing all nodes of the target type from thesource graph, with edges connecting all pairs of target nodes that sharea connection to a non-target node and edge weights representing thefrequency (or other aggregation) of such connections.

This approach can be used to project activity logs into actor-graphs orobject-graphs that encode the frequency of pairwise actions betweennodes of a single type. When rendered, such graphs can be qualitativelydecomposed into distinctive patterns of relationships with meaningfulbusiness interpretation (e.g., related people/services that might betargeted as a group). Use of graph-theoretic techniques to automaticallyidentify such meaningful substructures allows for the segmentation ofhigh-level metrics into dynamic cohorts of actors, actions, or objects.This in turn allows for the tracing of variation in high-level metricsto individual substructures and the diagnosis of these substructures asbeing the likely cause of the observed variation.

Each kind of substructure can itself be described using a range ofgraph-theoretic attributes corresponding to its size, shape, strength,and quality.

These attributes allow general relationships between activity structureand activity outcomes to be inferred; for example, through machinelearning, AI, or other forms of statistical inference. Such inferredrelationships, grounded in data, provide actionable suggestions forexperimental interventions aiming to transform both the structure andthe metrics of the activity for the better. The success of theseinterventions can then inform business strategy and direct theallocation of larger-scale business investments.

Example graph-theoretic structures, their associated attributes, and theimplied interventions are paths, connected components, communities, andcliques. In a path of nodes, each successive node is reachable byfollowing an edge from the previous node. The relationship between apair of nodes may be characterized by the length of the shortest pathbetween them or the number of distinct connecting paths of variouslengths. The relationship between a pair of paths can similarly becharacterized by the degree of separation or overlap between them.Possible interventions include incentives to reduce the shortest pathsbetween nodes known to be related or increasing the number of distinctpaths between them.

In a connected component of nodes, every node is reachable from everyother node by following the edges of a path. The size of a connectedcomponent may be measured by its number of nodes or edges and its shapeby its diameter (maximum eccentricity) and radius (minimumeccentricity), where the eccentricity of each node is its maximumdistance from any other node. Possible interventions includeincentivizing edges that bridge “disconnected” components to create alarger, single connected component, or which “round out” the componentby reducing the maximum distance between nodes.

In a community of nodes, there is a relatively greater density of edgesamong nodes inside the community than to nodes outside the community.The quality of a community partition of a graph is conventionallymeasured using the modularity function (the proportion of the edges thatfall within the given communities minus the expected proportion if edgeswere distributed at random) and the strength of the community by theaverage clustering coefficient of its nodes (the proportion of eachnode's connections that are connected to one another, out of allpossible connections). Possible interventions include incentives tocreate missing connections within communities or to bridge closelyrelated communities. Different community detection algorithms (e.g.,Louvain, VOS, and embedding plus clustering methods) may detectcommunities with different characteristics, such as partitioning thecenter and periphery of a graph versus two distinct sides (e.g., ingraphs of brain connectivity).

In a clique of nodes, each node is connected to each other node forminga fully-connected pattern of N(N−1)/2 edges, where N is the size of theclique. Possible interventions include incentivizing edges that completecliques by filling in missing edges, and growing cliques by connectingnew nodes to all existing nodes. The technique of link prediction mayalso be used to prioritize edges for targeting.

The same set of predictive graph techniques may be used in all instancesto help identify and prioritize relations of interest. For example,vertex nomination may be used to assign new nodes to existingcommunities based on their attributes, link prediction may be used toprioritize the addition of edges to paths, communities, or sub-cliques,graph matching may be used to suggest clique expansion or the structuralisomorphism of connected components, and dynamics forecasting may beused to predict future temporal dynamics based on historical activity.

The same graph-theoretic substructures and attributes that allowdecomposition of metrics within a time period also allow comparisonacross periods. Techniques here include creating a time series of graphrepresentations—each graph describing a time window such as a day orweek—and charting the variation in attributes and metrics both bysubstructure and over time. Such approaches help to reveal trends,cycles, and anomalies that may otherwise be masked by averaging over thewhole time period. Graphs from selected time windows may be comparedside-by-side by visually encoding nodes and edges (e.g., using a colorkey) based on the subset of time windows in which they occur. The sameencoding scheme may be applied to the graph spanning multiple timeperiods of interest such that dynamic activity may be represented in asingle, static graph. Alternative encoding schemes may highlightdifferent aspects of dynamic activity, e.g., coloring nodes and edgesusing a linear color scale based on their first or last appearance, orthe proportion of time windows in which they occur.

Certain examples described herein are a form of activity-basedsegmentation observed through the evidence of connected actions within apopulation of interdependent actors (e.g., obtained through telemetry,instrumentation, logging, crawling, etc.). Actions and actors/objectsare modeled as the edges and nodes of a directed graph respectively,typically with projection into undirected graphs with weightscorresponding to the strength of joint activity. Graph-theoreticattributes of the resulting graph structures (e.g., the modularity of apartitioning of the graph into communities of related nodes) are used todefine the activity-based segmentation.

Certain other examples relate to using activity-based segmentation toreveal topological and temporal variations in high-level metrics likedaily average users (DAU) and monthly active users (MAU). Suchstructured metrics allow drilldown into connected cohorts of actors orobjects and their patterns of activity over time. The result is thatstructural variations in high-level metrics can be revealed, analyzed,and acted upon.

Additional examples relate to using machine learning to model therelationship between the graph-theoretic attributes of segmentedsubgraphs (e.g., number of nodes/edges, average degree, subgraphdiameter, etc.) and their contribution to high-level metrics of interest(e.g., DAU, MAU). This result is an indication of how interventionsaiming to transform the structure of the graph might also transform bothactivity and metric for the better. These hypotheses can then be testedthrough targeted experiments (e.g., offering incentives to a group ofinfluencers).

Certain examples relate to varying the graph definition criteriasystematically to assess the sensitivity and stability of the resultinggraph structures and their attributes. The thresholds, time windows, andrelation types used to define graph edges are all significant.Thresholds that are too low will introduce noisy edges; thresholds thatare too high will lose meaningful structure. Defining edges fromactivity over too short a period may miss potential relationships ofinterest and lead to dramatic changes in graph structure from one periodto the next; defining edges from activity over too long a period mayinclude relationships no longer of relevance and fail to reveal dramaticchanges in activity within that period. When edges may representdifferent types of relationships (e.g., one-way versus bidirectionalcommunication in email, or document collaboration based on co-authoringin different application formats), the choice of edge definition maysimilarly result in graphs with very different qualities that mightsuggest very different interventions. To give a combined example, theimplicit graph based on any number of emails sent from one account toanother over a year will be very different to the implicit graph basedon ten or more bidirectional email exchanges within a week. A majorconsequence of using a graph formalism is that its flexibility allowsfor the dynamic generation of potential structured metrics. Users of theresulting metrics may attend to any subset of these according to theirneeds, using any substructures of any graphs to define meaningful,dynamic cohorts of actors or objects for comparison and monitoring overtime.

These various aspects described herein may be integrated in a softwareservice and deployed to the same infrastructure (e.g., the cloud) onwhich the activity data are logged and/or processed. This service may beoffered as an enhanced “network metrics” capability to existingthird-party users of cloud infrastructure as a way to extract additionalbusiness value from already-collected but not yet connected,instrumentation and telemetry data. The cloud infrastructure may enableactivity logging at an unprecedented level of detail and scale, frommonitoring of the processes and the media to the telemetric monitoringof product and service use. This data may be used to model activity in aconnected way that suggests a new class of structured metrics and theiruse to drive new levels of value.

FIG. 1 shows a diagram of a system environment 100 for collecting datain accordance with one example. System environment 100 may include userdevices 102, 104, and 106 connected via a network 120 to a set ofservers (or other types of devices) configured to provide variousapplications and services. As an example, the servers (which may behoused in a data center) may include an application server 130, an emailserver 140, and a file server 150. In an abstract, but still definite,sense the servers may provide compute and storage functionality to theusers of user devices 102, 104, and 106. The aspects described withrespect to FIG. 1 can be implemented in cloud computing environments. Inthis description, “cloud computing” is described as a model for enablingon-demand network access to a shared pool of configurable computingresources. For example, cloud computing can be employed in themarketplace to offer ubiquitous and convenient on-demand access to theshared pool of configurable computing resources. The shared pool ofconfigurable computing resources can be rapidly provisioned viavirtualization and released with low management effort or serviceprovider interaction, and then scaled accordingly. A cloud computingmodel may be composed of various characteristics such as, for example,on-demand self-service, broad network access, resource pooling, rapidelasticity, measured service, and so forth. A cloud computing model canalso expose various service models, such as, for example, Software as aService (“SaaS”), Platform as a Service (“PaaS”), and Infrastructure asa Service (“IaaS”). A cloud computing model can also be deployed usingdifferent deployment models such as private cloud, community cloud,public cloud, hybrid cloud, and so forth.

Still referring to FIG. 1, various telemetry agents may be configured tocollect, process, and manage telemetry data. These telemetry agents mayinclude monitoring agents, polling agents, notification agents, andcollectors. The telemetry agents may interface with the variousresources in system environment 100 via an application programminginterface (API). As an example, various telemetry agents 122, 124, and126 may be configured to monitor, poll, and notify telemetry data tocollectors, which may then pass the data for storage in one or moredatabases. Telemetry agents 122, 124, and 126 may be configured tomonitor, poll, and notify telemetry data to collectors corresponding toapplication server 130, email server 140, and file server 150. In acloud computing environment, each of these services may be offered via acollection of compute, networking, and storage resources. Telemetryagents 122, 124, and 126 may interface with any of these resources viaan appropriate API and collect telemetry data as part of the normaloperation of the cloud computing systems.

With continued reference to FIG. 1, user devices 102, 104, and 106 maybe any of a personal computing device, a laptop, a desktop, asmartphone, a tablet, or any other type of device that a user may use toaccess applications in a cloud computing environment. Each of userdevices 102, 104, and 106 may run applications installed in the cloudcomputing environment, including applications related to wordprocessing, presentations, graphics processing, database processing,email, messaging, collaboration and the like.

A logging service may also collect data from applications running in thecloud computing environment. The logging service (or services) maygenerate event files corresponding to application data and store them ina database. Streaming data may also be gathered using systems such asload balancers.

FIG. 2 shows a diagram of a cloud computing environment 200 forcollecting data in accordance with one example. Cloud computingenvironment 200 shows data collection for both compute resources andnon-compute resources. With respect to compute resources, cloudinfrastructure 212 may include activity logs 214. Cloud computingenvironment 200 may include host virtual machines, e.g., host VM 216,which may execute an operating system (e.g. OS 218). Diagnostic logs 230may collect diagnostics related data. Diagnostics data related to adeployed application on virtual machines in the cloud computing systemmay also be collected. This data may include event logs, performancecounters, crash dumps, or other diagnostics related information gatheredby the diagnostic service included as part of the cloud computingplatform. Applications may be executed in the operating systemenvironment and the data corresponds to such applications (e.g.,Application 220) may be collected in application logs 232. With respectto non-compute resources, such as storage resources, cloudinfrastructure 252 may include activity logs 254. The resources (e.g.,resource 250) may also contribute data to diagnostic logs 260. Variousmonitoring agents, similar to the ones described earlier, may be used tocollect the various types of data.

An active directory service (not shown) may also collect data forauditing purposes. As an example, the active directory service maycollect data relating to application usage and sign-in activity. Theapplication usage data may include application specific information. Asan example, with respect to file storage, the data may includeinformation concerning all activities related to file management. Theseactivities may include opening a file, modifying a file, storing a file,deleting a file, moving a file, or copying a file.

FIG. 3 is a block diagram of a system 300 for performing methodsassociated with the present disclosure in accordance with one example.System 300 may include a processor(s) 302, I/O component(s) 304, memory306, presentation component(s) 308, sensors 310, database(s) 312,networking interfaces 314, and I/O port(s), which may be interconnectedvia bus 320. Processor(s) 302 may execute instructions stored in memory306. I/O component(s) 304 may include components such as a keyboard, amouse, a voice recognition processor, or touch screens. Memory 306 maybe any combination of non-volatile storage or volatile storage (e.g.,flash memory, DRAM, SRAM, or other types of memories). Presentationcomponent(s) may include displays, holographic devices, or otherpresentation devices. Displays may be any type of display, such as LCD,LED, or other types of display. Sensors 310 may include telemetry orother types of sensors configured to detect, and/or receive, information(e.g., collected data). Sensors 310 may include sensors describedearlier or a subset of those sensors. Sensors 310 may be implemented inhardware, software, or a combination of hardware and software. Somesensors 310 may be implemented using a sensor API that may allow sensors310 to receive information via the sensor API.

Still referring to FIG. 3, database 312 may be used to store any of thedata collected or logged as part of the computing environment, asdescribed with respect to FIGS. 1 and 2. Database 312 may be implementedas a collection of distributed databases or as a single database.Network interfaces 314 may include communication interfaces, such asEthernet, cellular radio, Bluetooth radio, UWB radio, or other types ofwireless or wired communication interfaces. I/O port(s) may includeEthernet ports, Fiber-optic ports, wireless ports, or othercommunication ports. Although FIG. 3 shows system 300 as including acertain number of components arranged and coupled in a certain way, itmay include fewer or additional components arranged and coupleddifferently. In addition, the functionality associated with system 300may be distributed, as needed.

FIG. 4 shows a diagram of a memory 400 (e.g., memory 306 of FIG. 3)comprising modules with instructions for performing operationsassociated with system 300. Memory 400 may include telemetry sensors410, collector 420, extractor 430, graph inducer 440, and transformer450. Telemetry sensors 410 may include instructions, libraries, API, andother code for allowing receipt, formatting, and transmission of data,collected by the various telemetry agents and/or sensors describedearlier, to collector 420. Collector 420 may include instructions,libraries, API, and other code configured to collect and manage all ofthis data, which may be referred to as already-collected but not yetconnected data. Such data may include any of the data collected inactivity logs 214, diagnostic logs 230, and application logs 232associated with monitoring of applications and services in the cloudinfrastructure. Such data may further include activity logs 254 anddiagnostic logs 260 associated with resources being monitored in thecloud infrastructure. Without limitation, such data may includeapplication event logs, clickstream data, social network interactionsrelated data, e-commerce transactions related data, search enginerelated data, or any other data that may be collected during normaloperations of the cloud infrastructure, services, and applications.

With continued reference to FIG. 4, extractor 430 may includeinstructions, libraries, API, and other code configured to use dataextraction techniques to extract only data that relates to the dataneeded from the graph data embedding process. As an example, extractor430 may extract only that data which relates to a specific actor oractors, or actor type(s), or to a specific object or objects, or objecttype(s) with respect to a target activity. Graph inducer 440 mayrepresent the actions of actors as edges in a graph that connect nodes(representing the actors performing those actions and the objects beingacted upon). In certain examples, such edges are necessarily directedbecause the actor is performing the action on the object and notvice-versa. The resulting graph may also be heterogeneous in structure,representing relations between two distinct types of nodes (actors andobjects).

Still referring to FIG. 4, transformer 450 may include instructions,libraries, API, and other code configured to use graph embeddingmethod(s) to convert the discrete, high-dimensional graph representationinto a continuous, low-dimensional metric space. Transformer 450 may useany of a range of graph embedding techniques that can transform theedges of a graph into a metric space in which nodes with similarconnections are embedded at similar locations. This form ofdimensionality reduction has the advantage that discrete edge relationsare replaced by a continuous notion of relatedness based on distance(e.g., Euclidean) or similarity (e.g., Cosine), allowing the relatednessof any pair of nodes to be measured directly even if they lack aconnecting edge in the original graph structure. Such a transformationmay simplify and generalize the inference of the node communities andthe nearest neighbours.

Using systems and methods described in the present disclosure a user mayanalyze the data collected earlier to extract information concerning theactor, the action, the object, and the time representing a unit-levelcontribution to the overall activity. From these individual activitycontributions, one may induce a graph in which edges are created in oneof two typical ways depending on the node type of interest: (1) betweenactor nodes whenever both actors perform the same action of interest onthe same object in the same time window, with the edge weight reflectingthe aggregate (e.g., count) of shared actions across all objects, and(2) between object nodes whenever both objects receive the same actionof interest from the same actor in the same time window, with the edgeweight reflecting the aggregate (e.g., count) of shared actions acrossall actors.

From the same data, multiple graphs can be induced based on differentnodes, actions, and time windows of interest. Example time windows ofinterest include: (1) all time (i.e., since data collection began),suitable for when historic activity always remains equally relevant, (2)the last period of a given duration calculated in a rolling fashion(e.g., the last year), suitable for when recent activity is mostrelevant, and (3) all periods defined by a sliding and potentiallyoverlapping time window (e.g., a 4-week time window sliding in 2 weekincrements to create a 2-week overlap between adjacent time windows);suitable when change in activity is relevant.

To see the connected nature of such collaborative activity, one cancreate an undirected weighted graph in which nodes are individual users,edges represent evidence of collaboration within a defined time period,and edge weights correspond to the frequency of collaboration. As anexample, FIG. 5 shows a node-link graph 500 in which links representrelationships. Performing community detection (e.g., using Louvainmodularity optimization) over such a graph will reveal “communities” ofusers with strong collaboration among their members and relativelyweaker collaboration with users in other communities. FIG. 6 shows fourdifferent views (view 610, view 620, view 630, and view 640) of a graphthat reveals the communities of users with strong collaboration. Suchdynamic, activity-based segmentation of users into distinct communitiesover longer timeframes (e.g., based on historic data or a sliding windowof several months) allows finer-grained metrics (e.g., daily averageusers (DAU), weekly active users (WAU), and monthly active users (MAU))to be broken down and reported on a community-by-community basis, inways that reveal structural variation in use. Graph-theoretic attributesof these communities (e.g., number of users, number of collaborationlinks, average frequency per link, average collaborators per user, ratioof internal to external links, average clustering coefficient per user,etc.) can be used as features for machine learning that attempts tomodel the relationship between the structure of a community and theengagement that results. The same approach can also be applied to thecomplete user graph, with graph-theoretic attributes like the number andmodularity of communities used as additional features in the modeling ofoverall engagement metrics. Both nodes and edges may be cut from eachresulting graph to reduce noise and emphasize recurrent, meaningfulstructure. For example, edges may be cut if their weight is below athreshold, or edges may be cut in increasing weight order (or otherstatistics such as betweenness centrality or mutual information) untilthe reduced graph exhibits desirable statistical properties (such as atarget modularity score representing a clear partitioning of the graphinto communities of closely related nodes).

In certain examples consistent with the present disclosure, one may usea graph embedding method to convert the discrete, high-dimensional graphrepresentation into a continuous, low-dimensional metric space. Oneexample graph embedding method is node2vec, which uses random walks overthe graph to convert nodes into vector representations suitable for wordembedding using neural network models such as word2vec. In this example,the input weights to each neuron in the hidden layer may define adimension of the embedding space.

FIG. 7 shows a memory 700 including modules of instructions and data forgraph embedding related methods in accordance with one example. Theinstructions may be executed by processor 310 of FIG. 3 and memory 700may correspond to memory 306 of FIG. 3. Some other processors and/ormemories may also be used to store and execute the instructions. Memory700 may include a learning-based analyzer (LBA) 710, training data 720,machine learning (ML) models 730, graph data 740, embedding function750, and metric-space data 760. Although FIG. 7 shows instructions anddata organized in a certain way, the instructions and data may becombined or distributed in various ways. As an example, the data may beinput via databases coupled to high-speed memories and may also beoutput via high-speed memories before being stored in databases.

With continued reference to FIG. 7, learning-based analyzer (LBA) 710may implement a supervised learning algorithm that can be trained basedon input data and once it is trained it can make predictions orprescriptions based on the training. In this example, LBA 710 mayimplement techniques such as Linear Regression, Support Vector Machine(SVM) set up for regression, Random Forest set up for regression, orGradient-boosting trees set up for regression and neural networks.Linear regression may include modeling the past relationship betweenindependent variables and dependent output variables. Neural networksmay include artificial neurons used to create an input layer, one ormore hidden layers, and an output layer. Each layer may be encoded asmatrices or vectors of weights expressed in the form of coefficients orconstants that might have been obtained via off-line training of theneural network. Neural networks may be implemented as Recurrent NeuralNetworks (RNNs), Long Short Term Memory (LSTM) neural networks, or GatedRecurrent Unit (GRUs). All of the information required by a supervisedlearning-based model may be translated into vector representationscorresponding to any of these techniques. Taking the LSTM example, anLSTM network may comprise a sequence of repeating RNN layers or othertypes of layers. Each layer of the LSTM network may consume an input ata given time step, e.g., a layer's state from a previous time step, andmay produce a new set of outputs or states. In the case of using theLSTM, a single chunk of content may be encoded into a single vector ormultiple vectors. As an example, a node may be encoded as a singlevector. Each chunk may be encoded into an individual layer (e.g., aparticular time step) of an LSTM network. An LSTM layer may be describedusing a set of equations, such as the ones below:

i _(t)=σ(W _(xi) xt+W _(hi) h _(t−1) +W _(ci) c _(t−1) +b _(i)

f _(t)=σ(W _(xf) x _(t) +W _(hf) h _(t−1) +W _(cf) c _(t−1) +b _(f))

c _(t) =f _(t) c _(t−1) i _(t) tan h(W _(xc) x _(t) +W _(hc) h _(t−1) +b_(c))

o _(t)=σ(W _(xo) x _(t) +W _(ho) h _(t−1) +W _(co) c _(t) +b _(o))

h _(t) =o _(t) tan h(c _(t))

In this example, inside each LSTM layer, the inputs and hidden statesmay be processed using a combination of vector operations (e.g.,dot-product, inner product, or vector addition) or non-linearoperations, if needed.

Although FIG. 7 describes LBA 710 as comprising instructions, theinstructions could be encoded as hardware corresponding to an A/Iprocessor. In this case, some or all of the functionality associatedwith the learning-based analyzer may be hard-coded or otherwise providedas part of an A/I processor. As an example, A/I processor may beimplemented using an FPGA with the requisite functionality.

Training data 720 may be data that may be used to train a neural networkmodel or a similar machine learning model. In one example, training data720 may be used to train the machine learning model to minimize an errorfunction (or an objective function) associated with the conversion ofthe graph data to two-dimensional metrics data. ML models 730 mayinclude machine learning models that may prescribe strategies associatedwith the training and prediction. As an example, one example ML modelmay prescribe how many nodes should be traversed as part of random walksor some other algorithm. ML models 730 may further include models thatallow LBA 710 to model a relationship between graph attributesassociated with graph data and at least one higher-level metricassociated with a target activity. Graph attributes may include a sizeof the graph, a radius of the graph, a diameter of the graph, acentrality related measure associated with each of nodes of the graph, aconnectivity measure associated with the at least one graph, and adynamic measure associated with the graph. The higher-level metrics mayinclude any metrics that provide insights related to target activity. Asan example, the higher-level metrics may include scale metrics,stickiness metrics, and attrition metrics. Without limitation, scalemetrics may include daily active users (DAU), weekly active users (WAU),or monthly active users (MAU). Stickiness metrics may include ratios ofscale metrics to show how often users return to the service; forexample, a DAU/MAU ratio of 0.25 can be interpreted as users engagingwith the service on one day out of four on average. Attrition metricsmay include user retention percentages over various time periods (e.g.,Day 1, Day 2, Week 1). In one example, higher-level metrics may refer toany metrics that specifically relate to a target activity and are notsimply first order measurements associated with a graph.

In one example, the learning problem may be viewed as a maximumlikelihood optimization problem. Graph data 740 for a network may berepresented in terms of G=(V, E), where V corresponds to the set ofvertices and E corresponds to the set of edges for the graph G. Assumingthe mapping function (e.g., embedding function 750) corresponds tomapping nodes to vectors (e.g., metric space data 760 expressed in theform of two dimensions), the learning task may be to determine vectors(e.g., two dimensional vectors) that capture for every source node anetwork neighborhood of the node using a sampling strategy, which may beprovided by the ML model. The embedding function (e.g., embeddingfunction 750) may be an objective function that optimizes theprobability of observing network neighborhoods for a node. As used inthe node2vec algorithm, the sampling strategy may be a biased randomwalk procedure that could explore neighborhoods in both breadth-firstsampling (BFS) and depth-first sampling (DFS) fashion. For any graph,such as the one shown in FIG. 5, the node2vec algorithm may be used byimplementing three steps: 1) transition probabilities may beprecomputed, 2) random walk simulations may be performed, and 3)optimization may be performed using stochastic gradient descent (SGD).Once LBA 710 is trained, it could be used to represent each node in ametric space by using node2vec algorithm. While FIG. 7 shows oneapproach, other approaches may also be used. As an example, anotherclass of approaches, known as spectral graph embeddings, may performdimensionality reduction using eigen-decomposition of the graphLaplacian or adjacency matrix. The resulting approaches are known asLaplacian Spectral Embedding (LSE) and Adjacency Spectral Embedding(ASE) respectively. In one example, LSE may be used to convert graphdata into metric space data using adjacency matrix (the elements of thematrix indicate whether the pairs of vertices are adjacent or not in thegraph) and the degree centrality. In one example, the k eigenvectorscorresponding to the k lowest non-zero eigenvalues may define ak-dimensional embedding space. For dynamic graphs defined by differentgraph structures for each of multiple time windows, the OmnibusEmbedding (Omni) extension of the ASE allows the change in connectivityfor each node to be quantified.

A range of analytical questions about the activity (e.g., the activityof the users etc.) can be answered because of the metric spaces createdby these graph embeddings (e.g., the fact that the distances between anypair of points can be interpreted as “relatedness”). As an example, thenodes that are related to the node of interest may be determined byreturning the nearest neighbors to the node of interest in the embeddingspace, in a process known as vertex nomination. As another example, thestability of the connections of a node, community, group, or graph overtime may be determined by measuring the aggregate distance between nodesrepresenting the same underlying actor/object of interest in successivetime windows, then aggregating across all actors/objects of interest, ina process, which may be referred to as dynamics modeling.

FIG. 8 shows a flowchart 800 corresponding to a method including graphembedding in accordance with one example. Step 810 may includecollecting data for a target activity relating to an actor or an objectduring normal operation of a cloud computing platform to generate a setof already-collected but not yet connected data and storing the set ofthe already-collected but not yet connected data in the at least onememory. In one example, this step may be performed by collector 420,when instructions corresponding to collector 420 are executed by aprocessor (e.g., processor 302). In this example, already-collected butnot yet connected data may include any of the data collected in activitylogs 214, diagnostic logs 230, and application logs 232 associated withmonitoring of applications and services in the cloud infrastructure.Such data may further include activity logs 254 and diagnostic logs 260associated with resources being monitored in the cloud infrastructure.Without limitation, such data may include application event logs,clickstream data, social network interactions related data, e-commercetransactions related data, search engine related data, or any other datathat may be collected during normal operations of the cloudinfrastructure, services, and applications. Applications or services mayoutput such data by writing the data to files, e.g., batch files (e.g.,CSV files) stored in a storage associated with the cloud infrastructure.Applications or services may output such data by providing the data todatabases associated with the cloud infrastructure. Applications orservices may also stream data to a streaming or messaging service, whichin turn may transfer the data to a storage associated with the cloudinfrastructure. Streaming data may include telemetry data gathered bysensors that may collect data when certain events are triggered. Any ofthe sensors used for cloud telemetry may generate data that could bestored and made accessible as part of the already-collected but not yetconnected data. Streaming data may further include data generated by theInternet of Things (loT) devices connected or serviced by the cloudinfrastructure.

Step 820 may include using at least one processor, extracting at leastone of a first set of actor-related data, a second-set of object relateddata, and a third set of temporal data from the set of thealready-collected but not yet connected data representative of aunit-level contribution to the target activity. In one example, thisstep may be performed by extractor 430, when instructions correspondingto extractor 430 are executed by a processor (e.g., processor 302).Extractor 430 may use data extraction techniques to extract only datathat relates to the data needed from the graph data embedding process.As an example, extractor 430 may extract only that data which relates toa specific actor or actors, or actor type(s), or to a specific object orobjects, or object type(s) with respect to a target activity. Targetactivities may include activities related to a user's use ofapplications, services, or other aspects related to the cloud computingplatform. Additional examples of target activities and the extracteddata are provided with respect to certain applications of the graphembedding process in the context of certain use case scenarios. Suchextracted data may also be constrained based on time constraints, asdescribed earlier.

Step 830 may include using the at least one processor, generating graphdata for at least one graph having a plurality of nodes and a pluralityof edges using the set of the already-collected but not yet connecteddata, where each of the plurality of nodes corresponds to the actor orthe object, and where an attribute associated with each of the pluralityof edges corresponds to a measurement associated with the targetactivity during a temporal dimension of interest. In one example, thisstep may be performed by graph inducer 440, when instructionscorresponding to graph inducer 440 are executed by a processor (e.g.,processor 302). Graph inducer 440 may represent the actions of actors asedges in a graph that connect nodes (representing the actors performingthose actions and the objects being acted upon). In certain examples,such edges are necessarily directed because the actor is performing theaction on the object and not vice-versa. The resulting graph may also beheterogeneous in structure, representing relations between two distincttypes of nodes (actors and objects). For many forms of analysis,however, it may be beneficial to operate on an undirected graphstructure showing relations between nodes of only a single target type(e.g., actors or objects, but not both). Graph inducer 440 may create anew graph containing all nodes of the target type from a source graph,with edges connecting all pairs of target nodes that share a connectionto a non-target node and edge weights representing the frequency (orother aggregation) of such connections. This approach may be used toproject activity logs, which may be the extracted data, intoactor-graphs or object-graphs that encode the frequency of pairwiseactions between nodes of a single type. Depending on the node type ofinterest, graph inducer 440 may create in two ways. In one example, thegraph may be between actor nodes whenever both actors perform the sameaction of interest on the same object in the same time window, with theedge weight reflecting the aggregate (e.g., count) of shared actionsacross all objects. In another example, the graph may be between objectnodes whenever both objects receive the same action of interest from thesame actor in the same time window, with the edge weight reflecting theaggregate (e.g., count) of shared actions across all actors.

In terms of the temporal dimension, from the same already-collected butnot yet connected data, multiple graphs can be induced based ondifferent nodes, actions, and time windows of interest. Example timewindows of interest include: (1) all time (i.e., since data collectionbegan), which may be suitable for when historic activity always remainsequally relevant; (2) the last period of a given duration calculated ina rolling fashion (e.g., the last year), which may be suitable for whenrecent activity is most relevant; and (3) all periods defined by asliding and potentially overlapping time window (e.g., a 4-week timewindow sliding in 2 week increments to create a 2-week overlap betweenadjacent time windows), which may be suitable when change in the targetactivity is relevant.

With continued reference to FIG. 8, both nodes and edges may be cut fromeach resulting graph to reduce noise and emphasize recurrent, meaningfulstructure. For example, edges may be cut if their weight is below athreshold, or edges may be cut in increasing weight order (or otherstatistic such as betweenness centrality or mutual information) untilthe reduced graph exhibits desirable statistical properties (such as atarget modularity score representing a clear partitioning of the graphinto communities of closely related nodes).

Step 840 may include using the at least one processor, converting thegraph data into metric space data using a graph embedding process andstoring the metric space data in the at least one memory. In oneexample, this step may be performed by transformer 450, wheninstructions corresponding to transformer 450 are executed by aprocessor (e.g., processor 302). Transformer 450 may use a graphembedding method to convert the discrete, high-dimensional graphrepresentation into a continuous, low-dimensional metric space.Transformer 450 may use any of a range of graph embedding techniquesthat can transform the edges of a graph into a metric space in whichnodes with similar connections are embedded at similar locations. Thisform of dimensionality reduction has the advantage that discrete edgerelations are replaced by a continuous notion of relatedness based ondistance (e.g., Euclidean) or similarity (e.g., Cosine), allowing therelatedness of any pair of nodes to be measured directly even if theylack a connecting edge in the original graph structure. Such atransformation may simplify and generalize the inference of the nodecommunities and the nearest neighbours.

In one example, transformer 450 may use the node2vec graph embeddingprocess. The node2vec graph embedding process may require thespecification of several parameters, including: (1) window size, (2)in/out (P,Q) hyperparameters, (3) epochs, (4) walk length/number ofwalks per length, and (5) dimensions to embed to. Window size mayspecify how far does one look around a term within walk when performinga skipgram. Epochs may specify how many iterations to run through aspart of the node2vec process to stabilize the resulting embedding. Walklength/number of walks per node may have a significant impact on howsampling is performed. Dimensions to embed to specify the number ofdimensions used to represent a tensor as part of the embedded space. Thenumber of dimensions may be selected to high enough to describe theseparation between vertices adequately. As an example, the number ofdimensions may be 128. For a given task at hand, transformer 450 may beconfigured to tune these parameters accordingly. As an example, if thelocality of the graph embedding is the attribute that one wants to focuson then a smaller window size may be used. Epochs can be particularlyuseful to achieve a stability in the resulting embedding. Walk length isused to determine how much of the graph and which portions are sampled.Typically, the walk length needs to be kept sufficiently large to makesure one can see the coverage beyond just the local neighborhood. Inaddition, the walk length must be larger than the selected value for thewindow size parameter.

As part of the node2vec process, transformer 450 may represent each nodeof a graph by the neighborhood of other nodes that one is likely toreach by taking fixed-length random walks from that node. The transitionprobability for the next step in each random walk may be the product ofthe proportional weight of the outbound edge under consideration (fromthe total weight of all incident edges) and a parameter that may biasthe search towards a given balance of breadth-first and depth-firsttraversal. The final representation of each node's neighborhood may beaggregated from multiple random walks beginning at that node. In oneexample, transformer 450 may use these neighborhoods to train a neuralnetwork to predict nearby nodes. In this example, the prediction may beperformed using skipgram sampling, in which a single hidden-layerfeedforward neural network is trained to predict neighborhoods fromnodes through a process of stochastic gradient descent (SGD). Sincenearby nodes in the graph will have similar neighborhoods, the neuralnetwork will encode those nodes with similar weights. Transformer 450may map these weights into positions on spatial dimensions. The nodesthat are nearby in the graph in terms of link following will also benearby in the embedding space.

As part of step 840, other approaches may also be used. As an example,another class of approaches, known as spectral graph embeddings, mayperform dimensionality reduction using eigen-decomposition of the graphLaplacian or adjacency matrix. The resulting approaches are known as

Laplacian Spectral Embedding (LSE) and Adjacency Spectral Embedding(ASE) respectively. In these approaches, the k eigenvectorscorresponding to the k lowest non-zero eigenvalues define ak-dimensional embedding space. For dynamic graphs defined by differentgraph structures for each of multiple time windows, the OmnibusEmbedding (Omni) extension of the ASE allows the change in connectivityfor each node to be quantified. Although FIG. 8 shows certain number ofsteps being performed in a certain order, additional or fewer steps maybe performed in the same order or a different order.

FIG. 9 shows a process flow 900 in accordance with an example. Step 910may include identifying target metric(s) based on the connected actionsof interdependent actors with respect to an activity of interest. Inthis example, this step may include obtaining target metric(s) based ona use case scenario. Table 1 provides examples of activities ofinterest, which may be categorized into activities that relate tointeraction, production, consumption, or exchange. Document editing isan example activity related to the interaction activity. The targetmetrics for this activity may be daily average users (DAU), weeklyaverage users (WAU), or monthly active users (MAU).

TABLE 1 Graph Metric Implicit edge metric Inference Activity Exampleaverage definition drilldown technique Interaction Document DAU, Sameobject By Vertex editing WAU, community nomination MAU Production CaseCases Similar By Graph investiga- per objects connected matching tionperiod component Con- Content Click- Successive By Link sumptionbrowsing through objects walk/path prediction rate Exchange API serviceCalls/ Synchronous By clique Dynamics utilization dollars actionsforecasting per period

Step 920 may include creating an implicit activity graph in which theedges represent related actions of actors. Graph inducer 440 of FIG. 4may be used to perform this step. The implicit edges definition may bethe same objects for the document editing example. As shown in Table 1,implicit edges may also be defined based on similar objects, successiveobjects, or synchronous actions. As part of this step, machine learningmay be used to infer links. As an example, LBA 710 may access at leastone machine learning model (e.g., one of ML Models 730) to infer targetmetrics. As an example, in creating an activity graph relating to thetechnical support fraud, the phone numbers from where such fraudulentcalls are placed may be the nodes that link to the numbers that arebeing dialed from each of these phone numbers. A machine learning modelincluding fuzzy matching of area codes or other phone identifiers may beused to infer additional links among the various nodes in the activitygraph. In another example, online advertisements related to such scamsmay be analyzed. Thus, the online advertisements on web pages may beprocessed (e.g., using optical character recognition) to extract phonenumbers from the images of the offending web pages and connect phonenumbers to the web pages based on the visual similarity of the webpages. A machine learning model may be used to classify and identifyimages based on visual similarity.

Step 930 may include using graph-theoretic structures to supportactivity-based segmentation of the target metric(s). This step may beperformed by transformer 450 of FIG. 4. As an example, transformer 450may determine which nodes are related to a node of interest by returningthe nearest neighbors to the node of interest in the embedding spaceusing vertex nomination. Transformer 450 may also help determinemeaningful groups of related nodes by performing spatial clustering(e.g., DBSCAN) of the nodes in the embedding space. Transformer 450 mayalso help determine the stability of the connections of the node,including the stability of a community over time. In this example, usinga process referred to as dynamics modeling the stability may bedetermined by measuring the aggregate distance between nodesrepresenting the same underlying actor/object of interest in successivetime windows, then aggregating across all actors/objects of interest.

Step 940 may include inferring models of the relationships between thegraph-theoretic attributes of the graph substructures and theirrespective contribution to the target metric(s). A metric may beinterpreted according to the detected communities of the activity graph,revealing the structural distribution of the metric and supporting thediagnosis of problem communities that would benefit from intervention.As an example, a metric may be improved by using vertex nomination tosupport the target activity. For example, if the already-collected butnot yet connected data shows that an actor performs an action thatexpresses an interest in an object/actor, then the system could help theuser by using vertex nomination to recommend related objects/actors thatmight also be of interest (like query suggestions in a search engine orpeople suggestions in a social network). A metric may be predicted atthe graph and community levels using regression analysis over graphfeatures including basic graph measures (e.g., size, radius, diameter),centrality measures (e.g., degree centrality, betweenness centrality,closeness centrality, eigenvector centrality, percolation centrality),connectivity measures (e.g., clustering coefficient, density), anddynamics measures (e.g., Omni stability, scan statistic). Appropriatesupervised machine learning techniques for regression analysis includeArtificial Neural Networks, Support Vector Machines, k-Nearest Neighbors(k-NN), and linear regression. Predictions may be used to explore thepossible effects of activity interventions that transform the activitygraph in specific ways (e.g., increasing community sizes or density).LBA 710 of FIG. 7 may be used to perform at least some of thesepredictions. A metric driven by viral diffusion through the activitygraph may be modeled and predicted using feature diffusion over thegraph embedding. Possible interventions may include using predictions totarget nodes in ways that accelerates or slows the diffusion processthrough the activity graph.

Step 950 may include applying selected graph-theoretic changes to theinferred models. In this example, the selected graph-theoretic changesmay be applied in order to maximize the expected improvements to thetarget metric(s). Although FIG. 9 shows certain steps, additional orfewer steps may be performed.

FIGS. 10 and 11 show an example of generating a transition graph 1000based on already-collected but not yet connected data. In this example,already-collected but not yet connected data may include data gatheredby a search engine, such as Bing®, as part of the normal operation ofBing® as a service (provided via a cloud infrastructure). Transitiongraph 1100 may concern automotive entities, including automotive brands,such as Lamborghini, Maserati, Mercedes-Benz, Nissan, Honda, Toyota, GM,and Chrysler, etc. Each automotive entity may be represented by a nodeand transitions from one node to another may be recorded based on theactions of the users, such as querying for a car model, clicking on alink, or clicking on an advertisement. As an example, FIG. 10 shows asearch engine example 1000, having a user interface 1010, including asearch window 1012. Users may search for any of the models of cars,trucks, or SUVs offered by any of these entities. Users may click onadvertisements (e.g., 1014). Users may also click on any of theclickable links (e.g., 1016) or other user interface elements that maybe interactive. Successive actions by the users may add weight to thelinks between entities. As an example, if a user searches forLamborghini and then clicks on Mercedes-Benz, then the weight of thelink between the Lamborghini node and the Mercedes-Benz node mayincremented by a unit. Similarly, a search query for Mercedes-Benz afterthe user clicks on Lamborghini may result in incrementing of the weightby another unit. A subset of the graph formed by such nodes is shown astransition graph 1100 in FIG. 11. A community 1110 may represent a setof automotive entities that the users are more likely to switch among.Thus, a set of automotive entities that may be classified as Europeanluxury cars may correspond to the community 1110. Community 1110 mayfurther include sub-communities, such as sports cars 1120 and SUVs 1130.The sports cars sub-community may include nodes corresponding toautomotive entities, such as Lamborghini 1122 and Maserati 1124. Othercommunities may also be observed as part of transition graph 1100,including community 1140 and community 1150. Graph embedding techniquesmay be applied to transition graph 1100 to generate metric space data(for example, as described earlier with respect to FIG. 8), which may befurther analyzed.

In one example, a metric may be improved by using vertex nomination tosupport the target activity. For example, if the already-collected butnot yet connected data shows that an actor performs an action thatexpresses an interest in an object/actor, then the system could help theuser by using vertex nomination to recommend related objects/actors thatmight also be of interest (like query suggestions in a search engine orpeople suggestions in a social network). In an example, described withrespect to FIGS. 12, 13, and 14, recommendations may be provided as aresult of a search query submitted to a search engine. This example mayinclude determining a degree of similarity among products by leveragingthe already collected but not yet connected data, including, forexample, search engine query log data. Thus, in this example, the searchquery log data may be used to create a knowledge graph based on entitytransitions in search sessions. These transitions may include useractivities related to users clicking on suggestions or advertisements orusers performing follow-up search queries. Thus, as shown in FIG. 12,related entities view 1200 shows iPhone models that may have a highdegree of similarity. The models may be ranked based on the degree ofsimilarity. Data concerning frequency of the searches on the particularmodels may also be captured from the search engine query log data. Usingthis data, a search engine may, as shown in FIG. 13, in response to aquery 1320 entered via a search field 1310 (or provided via other means,such as voice input etc.) display similar products. Thus, a search query“apple iPhone xs max” may produce the display portion 1330 showingsimilar products that have the highest rank in terms of the similaritythat is determined using vertex nomination. As explained earlier, oncethe metric spaces have been created using the graph embeddings, thenearest neighbors to the node of interest in the space may be determinedusing vertex nomination. In one example, transformer 450 may includeinstructions configured to perform the steps associated with vertexnomination. The use of vertex nomination is not limited to productrecommendations; other application areas may include comparing airlinesetc.

As another example, FIG. 14 shows related entities view 1400 concerningairlines. Thus, a search query for Alaska Airlines may produce a list ofsuggested airlines based on the ranking shown in FIG. 14. This rankingmay similarly be based on the vertex nomination process applied to thesearch query log data for airlines' related searches. As before, thefrequency of the searches on each airline may also be captured.

FIGS. 15, 16, and 17 show insights that one may derive by applying thetechniques described in the present disclosure with respect to theadoption of a feature of an application or a service by the variouscommunities over time. As an example, FIG. 15 shows a graph 1510 thatrepresents the email exchanges among various users of an organization asan organization structure. Additional views, such as view 1520 of graph1510, show a community structure at a higher level of granularity. Eachof the nodes in view 1520 may represent a community of users that isinferred by analyzing the data corresponding to graph 1510 by usinggraph embedding techniques.

FIG. 16 shows a table 1600 illustrating adoption of a feature related toa product over time in accordance with one example. Column 1610 showsthe size of the various communities that have been identified using thegraph-theoretic techniques described herein. The adoption of the featureis shown over a time frame 1620 that extends from January 2017 to June2018. Different shadings (e.g., 1630, 1632, and 1634) represent theextent of the adoption of the feature being investigated. FIG. 17 showscommunity structure views 1710, 1720, 1730, and 1740 that show theadoption of the feature over time.

FIGS. 18-21 show the use of graph embedding techniques for workplaceanalytics. FIG. 18 shows a graph 1800 of various communities at aworkplace. Transformer 450 may determine meaningful groups of relatednodes (organized as communities in graph 1800) by performing spatialclustering (e.g., DBSCAN) of the nodes in the embedding space. Thus, inthis example, graph 1800 shows communities 1810, 1820, and 1830. Thelinks among communities indicate interactions among the communities. Themetric space data corresponding to graph 1800 obtained using thetechniques described earlier (e.g., node2vec) may be used to performadditional analytics. As an example, churn in the communities may beanalyzed by observing the monthly change in the community memberships. Ametric may be predicted at the graph and community levels usingregression analysis over graph features including basic graph measures(e.g., size, radius, diameter), centrality measures (e.g., degreecentrality, betweenness centrality, closeness centrality, eigenvectorcentrality, percolation centrality), connectivity measures (e.g.,clustering coefficient, density), and dynamics measures (e.g., Omnistability, scan statistic). Appropriate supervised machine learningtechniques for regression analysis include Artificial Neural Networks,Support Vector Machines, k-Nearest Neighbors (k-NN), and linearregression. Predictions may be used to explore the possible effects ofactivity interventions that transform the activity graph in specificways (e.g., increasing community sizes or density). A metric driven byviral diffusion through the activity graph can be modeled and predictedusing feature diffusion over the graph embedding. Possible interventionsmay include using predictions to target nodes in ways that accelerate orslow the diffusion process through the activity graph. The averagecommunity density could also be elevated to a high-level metric, whichitself may be capable of two forms of diagnostic drill-down. Transformer450 may perform a “structural drill-down” into the average density of aspecific community over time and a “dynamics drill-down” into thedensity of that specific community in a specific time window.Transformer 450 may also perform a “dynamics drill-down” into theaverage density of all communities within a specific time window and a“structural drill-down” into the density of a specific community in thatspecific time window.

The already-collected but not yet connected data may include emailtelemetry data, which may be processed to gather insights into theworkplace. As an example, FIG. 19 shows a conventional organizationdesign 1910 and the organization's functional reality 1920 based on agraph induced from email telemetry data, which may be a subset of thealready-collected but not yet connected data. Functional reality 1920may include nodes representing people (e.g., employees) that areconnected to each other via links that represent the email interactionsamong the people. The dark nodes in functional reality 1920 mayrepresent people (e.g., employees) who are connected outside of theirmanagement hierarchy. Light gray nodes may present people that areoperating within the silos of their reporting structure. Communities ofthe various types of nodes may be inferred using machine learning andthe techniques described earlier. Siloed working conditions (e.g.,represented by the communities or networks that mirror the hierarchy)may stifle innovation by fragmenting company knowledge and creating echochambers. This analysis thus may provide insights to managers, includingunderscoring those communities that are bureaucratic and limit theautonomy and the effectiveness of the informal networks.

Additional insights into the workplace may be obtained by additionalapplication of graph-theoretic techniques to the already-collected butnot yet connected data. As an example, FIG. 20 shows a framework 2000that illustrates certain attributes of a workplace. Framework 2000includes a horizontal axis representative of the fluidity of thecollaborative links and a vertical axis representative of the proportionof the links that are external to a group. The fluidity of thecollaborative links is based on the Omni score based on the embedding ofmultiple graphs into the same dimensions. The degree of the changes inthe collaborative links may represent focus shifting. The proportion ofthe links that are external to the group may indicate boundary crossingby employees. The proportion of the links that are external to the groupmay referred to as “freedom to collaborate across the organization” andis based on an analysis of the alignment between the community membersand the organizational hierarchy. The calculation involves computing theminimum spanning tree (MST) of all community members, adding all MSTnodes to a peer set, adding all peers of all MST nodes to the peer set(except for peers of the MST root node), and calculating alignment asthe ratio (community size)/(peer set size), and freedom to collaborateacross the organization as 1—alignment. Framework 2000 includes fourquadrants that classify the workplace into a colony 2010, hive 2020,nest 2030, or swarm 2040. Colony 2010 may relate to a workplacecommunity that has stable relationships within the group. Hive 2020 mayrepresent a workplace community that has agile reorganization within agroup boundary. Nest 2030 may relate to stable relationships spanninggroup boundaries. Swarm 2040 may relate to agile reorganization spanninggroup boundaries. Attributes, such as individual learning, grouplearning, and group execution, for these groups are also identified inFIG. 20. FIG. 21 shows another framework 2100 for investigating similarworkplace analytics. Framework 2100 also shows the degree of the changesin the collaborative links represented as focus shifting and theproportion of the links that are external to the group represented asboundary crossing by employees. Framework 2100 includes the same fourquadrants as in FIG. 20 that classify the workplace into a colony 2110,hive 2120, nest 2130, or swarm 2140. Groups 2112 and 2114 are shown ashaving the attributes associated with a colony; group 2122 is shown ashaving the attributes associated with a hive; group 2132 is shown as agroup having most of the employees having attributes associated with anest and having a small number of employees having attributes associatedwith a swarm.

Additional examples of analytics may relate to decreasing cybercrime.With many forms of cybercrime, the challenge is to find the connectionsbetween disparate pieces of evidence. A conventional investigativeworkflow is for investigators to manually piece together relatedevidence into cases built around a single identifying characteristic(e.g., a phone number), and for progress to be measured in terms ofcases processed over time. Such an approach can fail to integrate allrelated evidence in ways that misrepresent the potential impact oftackling each case. To see the connected nature of such cybercriminalactivity, we can create an undirected weighted graph in which nodes areevidential “fingerprints” associated with the perpetrator (e.g., adirect identifier like an alias, phone number or email address, or anindirect identifier like characteristic language or image use), edgesrepresent co-occurrence of fingerprints in individual pieces ofevidence, and edge weights correspond to the frequency of co-occurrence.The result is the creation of many separate subgraphs (connectedcomponents) that allow instances of cybercrime to be organized intodistinct operations. This organization of evidence aligns with the priornotion of a case and the metrics of cases tackled per unit time.

The graph-like organization of evidence also enables new,graph-theoretic metrics of crime sophistication to be used to prioritizeinvestigative and law enforcement efforts. Evidential fingerprints canbe measured and ranked not just by frequency but by degree (number ofconnected fingerprints) and operations sized not just by number offingerprints but by graph diameter (longest sequence of connectedfingerprints). Variations in the graph-theoretic attributes of scamoperations over time can also be used to monitor and describe thechanging tactics of cybercriminals in ways that would not otherwise bepossible. The technique of graph matching can also be used to identifyconnected components with similar structure that could indicategeneration by a common automated process.

Given the use of similarity measures to define graph edges, the use ofdifferent similarity thresholds will create different graph structures.Different graphs (and hence ranked lists of operations) can besystematically generated for different similarity thresholds as a wayfor users to quickly identify the most promising graphs for analysis.Over time, user feedback about the quality of matched fingerprints(e.g., by rejecting false matches) can be used to dynamically vary suchthresholds.

In conclusion, the present disclosure relates a method implemented by asystem comprising at least one processor and at least one memory. Themethod may include collecting data for a target activity relating to anactor or an object during normal operation of a cloud computing platformto generate a set of already-collected but not yet connected data andstoring the set of the already-collected but not yet connected data inthe at least one memory. The method may further include using the atleast one processor, extracting at least one of a first set ofactor-related data, a second set of object-related data, and a third setof temporal data from the set of the already-collected but not yetconnected data representative of a unit-level contribution to the targetactivity. The method may further include using the at least oneprocessor, generating graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor or the object, and where anattribute associated with each of the plurality of edges corresponds toa measurement associated with the target activity during a temporaldimension of interest. The method may further include using the at leastone processor, converting the graph data into metric space data using agraph embedding process and storing the metric space data in the atleast one memory.

The collecting the data for the target activity may comprise collectingapplication logs related data, activity logs related data, and streamingdata related to the actor or the object during the normal operation ofthe cloud computing platform. The converting the graph data into themetric space data using the graph embedding process may furthercomprise: (1) using spectral embedding, or (2) representing a nodeincluded in the graph data in terms of a neighborhood of other nodesthat are likely to be reached based on taking fixed-length random walksfrom the node. A final representation of the node may be based on anaggregation of a plurality of fixed-length random walks from the nodewhen the graph embedding process comprises representing the nodeincluded in the graph data in terms of the neighborhood of the othernodes.

The converting the graph data into the metric space data using the graphembedding process may further comprise performing dimensionalityreduction by replacing discrete edge relations with a distance-basedmeasure.

The already-collected but not yet connected data may comprise searchquery data collected by a search engine during a normal operation of thesearch engine. The already-collected but not yet connected data maycomprise a first log comprising records representing indicia of emailinteractions among a set of actors collected during a normal operationof an email service or a second log comprising records representingindicia of messaging interactions among a second set of actors during anormal operation of a collaboration application or a messagingapplication.

In another example, the present disclosure relates to a methodimplemented by a system comprising at least one processor and at leastone memory. The method may include collecting data for a target activityrelating to an actor or an object to generate a set of already-collectedbut not yet connected data and storing the set of the already-collectedbut not yet connected data in the at least one memory. The method mayfurther include using the at least one processor, extracting a set ofactor-related data from the set of the already-collected but not yetconnected data representative of a unit-level contribution to the targetactivity. The method may further include using the at least oneprocessor, generating graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor, and where an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest. The method may further include using the at leastone processor, converting the graph data into metric space data using agraph embedding process and storing the metric space data in the atleast one memory.

The collecting the data for the target activity may comprise collectingapplication logs related data, activity logs related data, and streamingdata related to the actor or the object during the normal operation ofthe cloud computing platform. The converting the graph data into themetric space data using the graph embedding process may furthercomprise: (1) using spectral embedding, or (2) representing a nodeincluded in the graph data in terms of a neighborhood of other nodesthat are likely to be reached based on taking fixed-length random walksfrom the node. A final representation of the node may be based on anaggregation of a plurality of fixed-length random walks from the nodewhen the graph embedding process comprises representing the nodeincluded in the graph data in terms of the neighborhood of the othernodes.

The converting the graph data into the metric space data using the graphembedding process may further comprise performing dimensionalityreduction by replacing discrete edge relations with a distance-basedmeasure.

The already-collected but not yet connected data may comprise searchquery data collected by a search engine during a normal operation of thesearch engine. The already-collected but not yet connected data maycomprise a first log comprising records representing indicia of emailinteractions among a set of actors collected during a normal operationof an email service or a second log comprising records representingindicia of messaging interactions among a second set of actors during anormal operation of a collaboration application or a messagingapplication.

In yet another example, the present disclosure relates to a systemcomprising at least one processor and at least one memory. The at leastone memory may include instructions configured to, when executed by theat least one processor, collect data for a target activity relating toan actor or an object to generate a set of already-collected but not yetconnected data and storing the set of the already-collected but not yetconnected data in the at least one memory. The at least one memory mayfurther include instructions configured to, when executed by the atleast one processor, extract a set of actor-related data from the set ofthe already-collected but not yet connected data representative of aunit-level contribution to the target activity. The at least one memorymay further include instructions configured to, when executed by the atleast one processor, generate graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor, and where an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest. The at least one memory may further includeinstructions configured to, when executed by the at least one processor,convert the graph data into metric space data using a graph embeddingprocess and storing the metric space data in the at least one memory.

The instructions configured to collect the data for the target activitymay further comprise instructions configured to collect application logsrelated data, activity logs related data, and streaming data related tothe actor or the object during the normal operation of the cloudcomputing platform. The instructions configured to convert the graphdata into the metric space data using the graph embedding process mayfurther comprise: (1) instructions for using spectral embedding, or (2)instructions configured to representing a node included in the graphdata in terms of a neighborhood of other nodes that are likely to bereached based on taking fixed-length random walks from the node. A finalrepresentation of the node may be based on an aggregation of a pluralityof fixed-length random walks from the node when the instructionsconfigured to representing a node included in the graph data in terms ofa neighborhood of other nodes are used.

The already-collected but not yet connected data may comprise searchquery data collected by a search engine during a normal operation of thesearch engine. The already-collected but not yet connected data maycomprise a first log comprising records representing indicia of emailinteractions among a set of actors collected during a normal operationof an email service or a second log comprising records representingindicia of messaging interactions among a second set of actors during anormal operation of a collaboration application or a messagingapplication.

In another example, the present disclosure relates to a methodimplemented by a system comprising at least one processor and at leastone memory. The method may include collecting data for a target activityrelating to an actor or an object during normal operation of a cloudcomputing platform to generate a set of already-collected but not yetconnected data and storing the set of the already-collected but not yetconnected data in the at least one memory.

The method may further include using the at least one processor,extracting at least one of a first set of actor-related data, a secondset of object-related data, and a third set of temporal data from theset of the already-collected but not yet connected data representativeof a unit-level contribution to the target activity. The method mayfurther include using the at least one processor, generating graph datafor at least one graph having a plurality of nodes and a plurality ofedges using the set of the already-collected but not yet connected data,where each of the plurality of nodes corresponds to the actor or theobject, and where an attribute associated with each of the plurality ofedges corresponds to a measurement associated with the target activityduring a temporal dimension of interest. The method may further includeusing the at least one processor, modeling a relationship between graphattributes associated with the graph data and at least one higher-levelmetric associated with the target activity. [000109] The collecting thedata for the target activity may comprise collecting application logsrelated data, activity logs related data, and streaming data related tothe actor or the object during the normal operation of the cloudcomputing platform. The modeling a relationship between graph attributesassociated with the graph data and the at least one higher-level metricassociated with the target activity may comprise using supervisedlearning to infer a relationship between at least one attributeassociated with the at least one graph and the at least one higher-levelmetric. The at least one attribute may be selected from a groupcomprising a size of the at least one graph, a radius of the at leastone graph, a diameter of the at least one graph, a centrality relatedmeasure associated with each of nodes of the at least one graph, aconnectivity measure associated with the at least one graph, aclustering measure associated with the at least one graph, a densitymeasure associated with the at least one graph, and a dynamic measureassociated with the at least one graph. The method may further compriseconverting the graph data into metric space data using a graph embeddingprocess prior to modeling the relationship between the graph attributesassociated with the graph data and the at least one higher-level metricassociated with the target activity.

The already-collected but not yet connected data may comprise searchquery data collected by a search engine during a normal operation of thesearch engine. The already-collected but not yet connected data maycomprise a first log comprising records representing indicia of emailinteractions among a set of actors collected during a normal operationof an email service or a second log comprising records representingindicia of messaging interactions among a second set of actors during anormal operation of a collaboration application or a messagingapplication.

In another example, the present disclosure relates to a methodimplemented by a system comprising at least one processor and at leastone memory. The method may include collecting data for a target activityrelating to an actor or an object to generate a set of already-collectedbut not yet connected data and storing the set of the already-collectedbut not yet connected data in the at least one memory. The method mayfurther include using the at least one processor, extracting a set ofactor-related data from the set of the already-collected but not yetconnected data representative of a unit-level contribution to the targetactivity. The method may further include using the at least oneprocessor, generating graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor, and where an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest. The method may further include using the at leastone processor, modeling a relationship between graph attributesassociated with the graph data and at least one higher-level metricassociated with the target activity.

The collecting the data for the target activity may comprise collectingapplication logs related data, activity logs related data, and streamingdata related to the actor or the object during the normal operation ofthe cloud computing platform. The modeling a relationship between graphattributes associated with the graph data and the at least onehigher-level metric associated with the target activity may compriseusing supervised learning to infer a relationship between at least oneattribute associated with the at least one graph and the at least onehigher-level metric. The at least one attribute may be selected from agroup comprising a size of the at least one graph, a radius of the atleast one graph, a diameter of the at least one graph, a centralityrelated measure associated with each of nodes of the at least one graph,a connectivity measure associated with the at least one graph, aclustering measure associated with the at least one graph, a densitymeasure associated with the at least one graph, and a dynamic measureassociated with the at least one graph. The method may further compriseconverting the graph data into metric space data using a graph embeddingprocess prior to modeling the relationship between the graph attributesassociated with the graph data and the at least one higher-level metricassociated with the target activity.

The already-collected but not yet connected data may comprise searchquery data collected by a search engine during a normal operation of thesearch engine. The already-collected but not yet connected data maycomprise a first log comprising records representing indicia of emailinteractions among a set of actors collected during a normal operationof an email service or a second log comprising records representingindicia of messaging interactions among a second set of actors during anormal operation of a collaboration application or a messagingapplication.

In yet another example, the present disclosure relates to a systemcomprising at least one processor and at least one memory. The at leastone memory may include instructions configured to, when executed by theat least one processor, collect data for a target activity relating toan actor or an object to generate a set of already-collected but not yetconnected data and storing the set of the already-collected but not yetconnected data in the at least one memory. The at least one memory mayfurther include instructions configured to, when executed by the atleast one processor, extract a set of actor-related data from the set ofthe already-collected but not yet connected data representative of aunit-level contribution to the target activity. The at least one memorymay further include instructions configured to, when executed by the atleast one processor, generate graph data for at least one graph having aplurality of nodes and a plurality of edges using the set of thealready-collected but not yet connected data, where each of theplurality of nodes corresponds to the actor, and where an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest. The at least one memory may further includeinstructions configured to, when executed by the at least one processor,model a relationship between graph attributes associated with the graphdata and at least one higher-level metric associated with the targetactivity.

The instructions configured to collect the data for the target activitymay further comprise instructions configured to collect application logsrelated data, activity logs related data, and streaming data related tothe actor or the object during the normal operation of the cloudcomputing platform. The instructions configured to model therelationship between the graph attributes associated with the graph dataand at least one higher-level metric associated with the target activitymay comprise supervised learning instructions configured to infer arelationship between at least one attribute associated with the at leastone graph and the at least one higher-level metric. The at least oneattribute may be selected from a group comprising a size of the at leastone graph, a radius of the at least one graph, a diameter of the atleast one graph, a centrality related measure associated with each ofnodes of the at least one graph, a connectivity measure associated withthe at least one graph, a clustering measure associated with the atleast one graph, a density measure associated with the at least onegraph, and a dynamic measure associated with the at least one graph.

The already-collected but not yet connected data may comprise searchquery data collected by a search engine during a normal operation of thesearch engine. The already-collected but not yet connected data maycomprise a first log comprising records representing indicia of emailinteractions among a set of actors collected during a normal operationof an email service or a second log comprising records representingindicia of messaging interactions among a second set of actors during anormal operation of a collaboration application or a messagingapplication.

It is to be understood that the methods, modules, and componentsdepicted herein are merely exemplary. Alternatively, or in addition, thefunctionality described herein can be performed, at least in part, byone or more hardware logic components. For example, and withoutlimitation, illustrative types of hardware logic components that can beused include Field-Programmable Gate Arrays (FPGAs),Application-Specific Integrated Circuits (ASICs), Application-SpecificStandard Products (ASSPs), System-on-a-Chip systems (SOCs), ComplexProgrammable Logic Devices (CPLDs), etc. In an abstract, but stilldefinite sense, any arrangement of components to achieve the samefunctionality is effectively “associated” such that the desiredfunctionality is achieved. Hence, any two components herein combined toachieve a particular functionality can be seen as “associated with” eachother such that the desired functionality is achieved, irrespective ofarchitectures or inter-medial components. Likewise, any two componentsso associated can also be viewed as being “operably connected,” or“coupled,” to each other to achieve the desired functionality.

The functionality associated with some examples described in thisdisclosure can also include instructions stored in a non-transitorymedia. The term “non-transitory media” as used herein refers to anymedia storing data and/or instructions that cause a machine to operatein a specific manner. Exemplary non-transitory media includenon-volatile media and/or volatile media. Non-volatile media include,for example, a hard disk, a solid-state drive, a magnetic disk or tape,an optical disk or tape, a flash memory, an EPROM, NVRAM, PRAM, or othersuch media, or networked versions of such media. Volatile media include,for example, dynamic memory such as DRAM, SRAM, a cache, or other suchmedia. Non-transitory media is distinct from, but can be used inconjunction with transmission media. Transmission media is used fortransferring data and/or instruction to or from a machine. Exemplarytransmission media include coaxial cables, fiber-optic cables, copperwires, and wireless media, such as radio waves.

Furthermore, those skilled in the art will recognize that boundariesbetween the functionality of the above described operations are merelyillustrative. The functionality of multiple operations may be combinedinto a single operation, and/or the functionality of a single operationmay be distributed in additional operations. Moreover, alternativeembodiments may include multiple instances of a particular operation,and the order of operations may be altered in various other embodiments.

Although the disclosure provides specific examples, variousmodifications and changes can be made without departing from the scopeof the disclosure as set forth in the claims below. Accordingly, thespecification and figures are to be regarded in an illustrative ratherthan a restrictive sense, and all such modifications are intended to beincluded within the scope of the present disclosure. Any benefits,advantages, or solutions to problems that are described herein withregard to a specific example are not intended to be construed as acritical, required, or essential feature or element of any or all theclaims.

Furthermore, the terms “a” or “an,” as used herein, are defined as oneor more than one. Also, the use of introductory phrases such as “atleast one” and “one or more” in the claims should not be construed toimply that the introduction of another claim element by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim element to inventions containing only one such element,even when the same claim includes the introductory phrases “one or more”or “at least one” and indefinite articles such as “a” or “an.” The sameholds true for the use of definite articles.

Unless stated otherwise, terms such as “first” and “second” are used toarbitrarily distinguish between the elements such terms describe. Thus,these terms are not necessarily intended to indicate temporal or otherprioritization of such elements.

What is claimed:
 1. A method implemented by a system comprising at leastone processor and at least one memory, the method comprising: collectingdata for a target activity relating to an actor or an object duringnormal operation of a cloud computing platform to generate a set ofalready-collected but not yet connected data and storing the set of thealready-collected but not yet connected data in the at least one memory;using the at least one processor, extracting at least one of a first setof actor-related data, a second set of object-related data, and a thirdset of temporal data from the set of the already-collected but not yetconnected data representative of a unit-level contribution to the targetactivity; using the at least one processor, generating graph data for atleast one graph having a plurality of nodes and a plurality of edgesusing the set of the already-collected but not yet connected data,wherein each of the plurality of nodes corresponds to the actor or theobject, and wherein an attribute associated with each of the pluralityof edges corresponds to a measurement associated with the targetactivity during a temporal dimension of interest; and using the at leastone processor, modeling a relationship between graph attributesassociated with the graph data and at least one higher-level metricassociated with the target activity.
 2. The method of claim 1, whereinthe collecting the data for the target activity comprises collectingapplication logs related data, activity logs related data, and streamingdata related to the actor or the object during the normal operation ofthe cloud computing platform.
 3. The method of claim 1, wherein themodeling a relationship between graph attributes associated with thegraph data and the at least one higher-level metric associated with thetarget activity comprises using supervised learning to infer arelationship between at least one attribute associated with the at leastone graph and the at least one higher-level metric.
 4. The method ofclaim 3, wherein the at least one attribute is selected from a groupcomprising a size of the at least one graph, a radius of the at leastone graph, a diameter of the at least one graph, a centrality relatedmeasure associated with each of nodes of the at least one graph, aconnectivity measure associated with the at least one graph, aclustering measure associated with the at least one graph, a densitymeasure associated with the at least one graph, and a dynamic measureassociated with the at least one graph.
 5. The method of claim 1 furthercomprising the converting the graph data into metric space data using agraph embedding process prior to the modeling the relationship betweenthe graph attributes associated with the graph data and the at least onehigher-level metric associated with the target activity.
 6. The methodof claim 1, wherein the already-collected but not yet connected datacomprises search query data collected by a search engine during a normaloperation of the search engine.
 7. The method of claim 1, wherein thealready-collected but not yet connected data comprises at least one of afirst log comprising records representing indicia of email interactionsamong a first set of actors collected during a normal operation of anemail service or a second log comprising records representing indicia ofmessaging interactions among a second set of actors during a normaloperation of a collaboration application or a messaging application. 8.A method implemented by a system comprising at least one processor andat least one memory, the method comprising: collecting data for a targetactivity relating to an actor or an object to generate a set ofalready-collected but not yet connected data and storing the set of thealready-collected but not yet connected data in the at least one memory;using the at least one processor, extracting a set of actor-related datafrom the set of the already-collected but not yet connected datarepresentative of a unit-level contribution to the target activity;using the at least one processor, generating graph data for at least onegraph having a plurality of nodes and a plurality of edges using the setof the already-collected but not yet connected data, wherein each of theplurality of nodes corresponds to the actor, and wherein an attributeassociated with each of the plurality of edges corresponds to ameasurement associated with the target activity during a temporaldimension of interest; and using the at least one processor, modeling arelationship between graph attributes associated with the graph data andat least one higher-level metric associated with the target activity. 9.The method of claim 8, wherein the collecting the data for the targetactivity comprises collecting application logs related data, activitylogs related data, and streaming data related to the actor or the objectduring the normal operation of the cloud computing platform.
 10. Themethod of claim 8, wherein the modeling the relationship between thegraph attributes associated with the graph data and the at least onehigher-level metric associated with the target activity comprises usingsupervised learning to infer a relationship between at least oneattribute associated with the at least one graph and the at least onehigher-level metric.
 11. The method of claim 10, wherein the at leastone attribute is selected from a group comprising a size of the at leastone graph, a radius of the at least one graph, a diameter of the atleast one graph, a centrality related measure associated with each ofnodes of the at least one graph, a connectivity measure associated withthe at least one graph, a clustering measure associated with the atleast one graph, a density measure associated with the at least onegraph, and a dynamic measure associated with the at least one graph. 12.The method of claim 8 further comprising converting the graph data intometric space data using a graph embedding process prior to modeling therelationship between the graph attributes associated with the graph dataand the at least one higher-level metric associated with the targetactivity.
 13. The method of claim 8, wherein the already-collected butnot yet connected data comprises search query data collected by a searchengine during a normal operation of the search engine.
 14. The method ofclaim 8, wherein the already-collected but not yet connected datacomprises at least one of a first log comprising records representingindicia of email interactions among a set of actors collected during anormal operation of an email service or a second log comprising recordsrepresenting indicia of messaging interactions among a second set ofactors during a normal operation of a collaboration application or amessaging application.
 15. A system comprising at least one processorand at least one memory, the at least one memory comprising:instructions configured to, when executed by the at least one processor,collect data for a target activity relating to an actor or an object togenerate a set of already-collected but not yet connected data andstoring the set of the already-collected but not yet connected data inthe at least one memory; instructions configured to, when executed bythe at least one processor, extract a set of actor-related data from theset of the already-collected but not yet connected data representativeof a unit-level contribution to the target activity; instructionsconfigured to, when executed by the at least one processor, generategraph data for at least one graph having a plurality of nodes and aplurality of edges using the set of the already-collected but not yetconnected data, wherein each of the plurality of nodes corresponds tothe actor, and wherein an attribute associated with each of theplurality of edges corresponds to a measurement associated with thetarget activity during a temporal dimension of interest; andinstructions configured to, when executed by the at least one processor,model a relationship between graph attributes associated with the graphdata and at least one higher-level metric associated with the targetactivity.
 16. The system of claim 15, wherein the instructionsconfigured to collect the data for the target activity further comprisesinstructions configured to collect application logs related data,activity logs related data, and streaming data related to the actor orthe object during the normal operation of the cloud computing platform.17. The system of claim 15, wherein the instructions configured to modelthe relationship between the graph attributes associated with the graphdata and at least one higher-level metric associated with the targetactivity comprise supervised learning instructions configured to infer arelationship between at least one attribute associated with the at leastone graph and the at least one higher-level metric.
 18. The system ofclaim 17, wherein the at least one attribute is selected from a groupcomprising a size of the at least one graph, a radius of the at leastone graph, a diameter of the at least one graph, a centrality relatedmeasure associated with each of nodes of the at least one graph, aconnectivity measure associated with the at least one graph, aclustering measure associated with the at least one graph, a densitymeasure associated with the at least one graph, and a dynamic measureassociated with the at least one graph.
 19. The system of claim 15,wherein the already-collected but not yet connected data comprisessearch query data collected by a search engine during a normal operationof the search engine.
 20. The system of claim 15, wherein thealready-collected but not yet connected data comprises a log comprisingrecords representing indicia of email interactions among a set of actorscollected during a normal operation of an email service.